Technical Field
The disclosed embodiments relate in general to music interface design and, more specifically, to systems and methods for playing virtual music instrument through tracking of fingers with coded light.
Description of the Related Art
Many sensing technologies have been explored for music interface design. Visual tracking approach where the camera is the only or the main sensor is generally considered to be a dominating technique, which covers a wide field of applications. This method enables many different objects or body parts to be tracked independently without other special equipment, see, for example, Kolesnik, P. 2004. Conducting gesture recognition, analysis and performance system. Master Thesis. McGill University. Main disadvantages of this technique are the significant power consumption and required high storage capacity, which may be challenging to implement in practice.
Other sensor-based systems can also provide a greater range of data and make very expressive musical controllers. For example, magnetic tracking described in Ilmonen, T., and Takala, T., 1999, Conductor following with artificial neural networks, accelerometer tracking described in Varni, G., Dubus, G., Oksanen, S., Volpe, G., Fabiani, M., Bresin, R., Kleimola, J., Valimaki, V., and Camurri, A. 2012. Interactive sonification of synchronisation of motoric behaviour in social active listening to music with mobile devices. Journal on Multimodal User Interfaces, 5(3-4), 157-173, and gyroscope tracking described in Dillon, R., Wong, G., and Ang, R. 2006. Virtual orchestra: An immersive computer game for fun and education. In Proceedings of the 2006 international conference on Game research and development, 215-218, have all been explored in previous studies. However, most of them are susceptible to a fair amount of unpredictable noise either from the sensing system itself or the surrounding environment.
In view of the above and other shortcomings of the conventional tracking technology, new and improved systems and methods for finger tracking are needed that could be used in music interface designs for enabling users to play virtual music instruments.